Since 1997, four national studies conducted by the 
U.S. Department of Education have been completed on 
the Federal TRIO programs. Data from these studies 
demonstrate that participation in a TRIO program has 
a significant impact on the educational outcomes of 
low-income, first-generation students 
and students with disabilities. 
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More Detailed Findings 2 

Retention 

Compared to similarly qualified students 
in the control group, SSS students were: 

• 12% more likely to be retained to 
the second year (67% vs. 60%), and 

• 23% more likely to be retained to 
the third year (49% vs. 40%). 

Credits Earned 

Compared to similarly qualified 
students in the control group, 

SSS students earned: 

• 6% more credits in the first year, 

• 4% more credits in the second year, 
and 

• 4% more credits in the third year. 

GPA 

Compared to similarly qualified students 
in the control group, SSS students’ 

GPAs were: 

• 7% higher in the first year, 

• 5% higher in the second year, and 

• 4% higher over three years 
(cumulative). 



Student Support Services 

In 1997, Westat, Inc. completed and submitted an assessment report 1 
to the U.S. Department of Education on the impact of Student Support 
Services (SSS). The report’s findings suggest that SSS has a statistically 
significant positive effect on student success, persistence, and retention. 
These findings conclusively indicate that SSS is an effective program. 
When compared to similarly qualified students, who did not participate in 
the program, SSS participants were more likely to: 

1. Remain enrolled in higher education, 

2. Accrue MORE COLLEGE CREDITS, AND 

3 . Earn higher grade point averages (GPAs). 

The study also found that the size of program impact was directly and 
positively related to the level of program participation. Stated otherwise, 
the effects of the program were found to be more profound as students’ 
individual participation increased. In addition, the effect of SSS on 
outcomes was determined to be related to specific services received- 
peer tutoring in the first year showed the greatest positive effect. 

Methods 

This study utilized a quasi-experimental design and regression analyses to 
assess the impact of Student Support Services. A total of 5,800 students 
at 47 postsecondary institutions were tracked over the course of a three 
year period. A comparison group of 2,900 students was constructed 
to match the demographic and educational profiles of 2,900 first-year 
student SSS participants who enrolled during the 1991-92 academic year. 
Subsequent follow-up studies championing the effectiveness of SSS have 
been completed; however, these reports were not released by the 
George W. Bush Administration and have yet to be published. 

Program Description 

Student Support Services programs provide tutoring, counseling, and 
remedial instruction to low-income, first-generation college students and 
students with disabilities. These programs enable students to successfully 
begin their college careers, persist in their studies, and, ultimately, earn 
bachelor’s degrees. Currently, there are more than 945 programs serving 
nearly 200,000 students nationwide. 



1 Chaney, B., Muraskin, L., Cahalan, M., & Rak, R. (1997). National study of Student Sup- 
port Services: Third-year longitudinal study of results and program implementation 
study update. Washington, DC: U.S. Department of Education. 

2 Data presented throughout this brief are a percent change in the mean, performance 
rate, or participation rate of students in the treatment group relative to students in 
the control group. 



Talent Search 



In 2006, Mathematica Policy Research, Inc. (MPR) completed and 
submitted an assessment report 3 to the U.S. Department of Education on 
the impact of Talent Search on secondary and postsecondary outcomes 
in Texas, Indiana, and Florida. When compared to similarly qualified 
students, who did not participate in the program, statistically significant 
findings revealed that, across all states, Talent Search participants 
were more likely to: 

1. Apply for financial assistance, and 

2. Enroll in postsecondary education. 

This impact is particularly notable because the cost per student for 
Talent Search is less than $380. 

Methods 

This study utilized a quasi-experimental design and regression analyses 
to assess the effectiveness of Talent Search projects in Texas, Indiana, and 
Florida. These states were specifically chosen for inclusion in the study 
because administrative educational data systems allowed for student 
follow-up and the selection of appropriate control groups. Data from 60% 
of Talent Search projects in these states were used in the study. Talent 
Search data merged with federal financial aid and state data to assess the 
impact of Talent Search participation on select outcomes. 

Program Description 

Talent Search is a college access program that serves low-income youth in 
grades six through twelve. In addition to counseling, participants receive 
information about college admission requirements, scholarships, and 
various student financial aid programs. Talent Search also provides 
low-income students and their families with assistance in choosing a 
college, applying for college, and applying for financial aid. This early 
intervention program helps young people better understand their 
educational opportunities and options. More than 360,000 students are 
enrolled in 466 programs across the country. 



More Detailed Findings by State 

Texas 

Compared to similarly qualified students 
in the control group, Talent Search 
students were: 

• 12% more likely to attain a high 
school diploma (86% vs. 77%), 

• 77% more likely to be first time 
financial aid applicants 

(62% vs. 35%), and 

• 52% more likely to enroll full-time 
in a Texas public college 

(38% vs. 25%). 

Indiana 

Compared to similarly qualified students 
in the control group, Talent Search 
students were: 

• 31% more likely to be a first time 
financial aid applicant 

(59% vs. 45%), and 

• 8% more likely to enroll in an 
Indiana public college 

(56% vs. 52%). 

Florida 

Compared to similarly qualified students 
in the control group, Talent Search 
students were: 

• 20% more likely to attain a high 
school diploma (84% vs. 70%), 

• 58% more likely to be first time 
applicants for financial aid 
(52% vs. 33%), and 

• 42% more likely to immediately 
enroll in a Florida public college 
(51% vs. 36%). 

Variations among states could be due to 
differences in state and privately funded 
college access services available locally. 



3 Constantine, ). M, Seftor, N. S., Martin, E. S., Silva, T., & Myers, D. (2006). A study of 
the effect of the Talent Search program on secondary and postsecondary outcomes 
in Florida, Indiana, and Texas: Final report from phase II of the national evaluation. 
Washington, DC: U.S. Department of Education. 




More Detailed Findings 

Compared to control group participants, 
Upward Bound students were: 

• 50% more likely to attain a 
bachelor’s degree (21.1% vs. 14.1%), 

• 19% more likely to attain any 
postsecondary degree or credential 
(49.3% vs. 41.5%), and 

• 22% more likely to apply for 
student financial aid 
(75.0% vs. 61.7%). 



Upward Bound 

In 2009, Mathematica Policy Research, Inc. (MPR) released and submitted 
an assessment report 4 to the U.S. Department of Education on the impact 
of Upward Bound on postsecondary outcomes. Dr. Margaret Cahalan, 
the Project Officer for the U.S. Department of Education that oversaw 
the technical competence of the MPR report, conducted a follow-up 
analysis 5 of the same data and found fatal methodological flaws in the 
original MPR study. Dr. Cahalan’s re-analysis determined Upward Bound 
had statistically significant and substantive positive effects on three key 
outcomes. Specifically, the findings revealed that when compared to an 
equally qualified comparison group, Upward Bound participants were 
more likely to: 

1. Complete a college degree, 

2. Enroll in college, and 

3. Apply FOR STUDENT FINANCIAL AID. 

Methods 

The National Evaluation of Upward Bound (UB) is a nationally- 
representative random assignment study that followed a multi-grade 
cohort from 1992-93 to 2003-04 to evaluate the impact of Upward 
Bound. About 1,500 students nationwide were randomly assigned to the 
treatment group and about 1,300 students who were not selected for 
Upward Bound were assigned to the control group. These students were 

from 67 of the 395 projects that 
had been operating for three years 
by October 1992. 




4 Seftor, N. S., Mamun, A., Schirm, A. (2009). The impacts of regular Upward Bound on 
postsecondary outcomes 7-9 years after scheduled high school graduation. Princeton, 
N): Mathematica Policy Research, Inc. 

5 Cahalan, M. (in press). Do the conclusions change? The case of Upward Bound, 
correcting for study error in the 1992-2004 national evaluation of Upward Bound. 
Washington, DC: The Pell Institute for the Study of Opportunity in Higher Education 
(publication will be posted on The Pell Institute’s website - www.pellinstitute.org). 




Technical Issues in MPR Study Design 

The major technical issues in the MPR study were: (1) 
extreme unequal weighting, (2) treatment-control group 
non-equivalency, (3) survey non-response bias, (4) lack 
of standardization for expected high school graduation 
year, and (5) service submission and dropout issues. To 
correct these issues, Dr. Cahalan’s study used a complete 
longitudinal file, included administrator record data 
to correct for survey non-response bias, standardized 
expected high school graduation dates in deriving the 
outcome variables, conducted sensitivity analysis of 
weighting and outlier issues, and additional quasi- 
experimental design. 

The Upward Bound Program 

Upward Bound is an early intervention program that 
helps low-income and potential first-generation college 
students prepare for higher education. Upward Bound 
projects bring high school students to college campuses 
after school, on the weekends, and during the summer to 
receive instruction in mathematics, laboratory sciences, 
composition, literature, and foreign languages. Tutoring, 
counseling, mentoring, cultural enrichment, and work- 
study programs are additional components to the Upward 
Bound program. Currently, there are 964 programs serving 
more than 65,000 students across the country. 






Upward Bound Math-Science 



In 2008, Mathematica Policy Research, Inc. (MPR) released and submitted 
an assessment report 6 to the U.S. Department of Education on the 
impact of Upward Bound Math-Science on postsecondary completion, 
performance, enrollment, and participation in math and science 
coursework both at the high school and postsecondary level. The study 
found that Upward Bound Math-Science had statistically significant and 
substantive positive effects on five outcomes. Specifically, the findings 
revealed that when compared to equally qualified students in the 
comparison group, Upward Bound Math-Science participants were more 
likely to: 

1. Complete a four-year degree in math or science, 

2. Enroll in selective four-year institutions, 

3. Major in math or science disciplines, 

4. Take chemistry and physics coursework in high school, and 

5. Improve grades in math and science. 



6 Olsen, R., Seftor, N., Silva, T., Myers, D., DesRoches, D., & Young, J. (2008). 
Upward Bound Math-Science: Program description and interim impacts. 
Princeton, NJ: Mathematica Policy Research, Inc. 







Methods 

This impact analysis compared a sample of 1,500 Upward Bound Math- 
Science participants with a comparison group of students who applied to 
Upward Bound programs, but did not participate in Upward Bound Math- 
Science. In addition to displaying similar demographic characteristics, 
individuals in the comparison group possessed motivation levels 
similar to the sample, evidenced through their Upward Bound program 
application completion. MPR minimized selection bias by statistically 
controlling for any small remaining differences in the characteristics 
between groups. 

The Upward Bound Math-Science Program 

Upward Bound Math-Science is an early intervention program that works 
with low-income and potential first-generation college students who 
show an interest in math and science. Upward Bound Math-Science 
projects provide intensive instruction and innovative curricula during 
both the academic year and summer in math and science, as well as in 
other subjects such as computer technology, English, foreign languages, 
and study skills. The program also provides academic and social support 
through tutoring, counseling, mentoring, cultural enrichment, field trips, 
and financial aid application assistance. There are currently 115 programs 
serving 6,250 students throughout the country. 



More Detailed Findings 

Compared to similarly qualified 
students in the control group, 

UBMS students were: 

• Twice as likely to attain a BA in 
math and science (12% vs. 6%), 

• 44% more likely to enroll in 
selective four-year colleges and 
universities (33% vs. 23%), 

• 43% more likely to select math or 
science as a college major 

(33% vs. 23%), 

• 35% more likely to take high school 
physics (58% vs. 43%), and 

• 13% more likely to take high school 
chemistry (88% vs. 78%). 




About The Pell Institute 

The Pell Institute for the Study of Opportunity in Higher Education conducts and disseminates 
research and policy analysis to encourage policymakers, educators, and the public to improve 
educational opportunities and outcomes for low-income, first-generation students and students 
with disabilities. The Pell Institute is the first research institute to specifically examine the 
challenges affecting educational opportunity for these growing populations. 



Pell Institute Staff 
and Senior Scholars 

Chandra Taylor Smith 

Director and Vice President for Research 
Andrew Howard Nichols 
Senior Research Analyst 
Abby Miller 

Research and Project Manager 
Ryan Miller 

Research Analyst 
Adolfo Bermeo 
Senior Scholar 
Thomas Mortenson 
Senior Scholar 
Lana Muraskin 
Senior Scholar 
Congressman Louis Stokes 
Senior Scholar 
Vincent Tinto 
Senior Scholar 



Advisory Board Members 

Sonya Anderson 
National Director, 

The First Five Years Fund 
Estela Mara Bensimon 
Professor and Director, 

Center for Urban Education, 

Rossier School of Education, 
University of Southern California 
Karen Boran 

Director, IDS Curriculum, 

High School Teaching and Learning 
Chicago Public Schools 
Betsy Brand 

Director, American Youth Policy Forum 
Alberto Cabrera 

Professor, Department of Education 
Leadership, Higher Education and 
International Education, University of 
Maryland, College Park 
Brenda Dann-Messier 

President/CEO, Dorcas Place Adult 
and Family Learning Center 
Heather Eggins 

Visiting Professor, Institute for Access 
Studies, Staffordshire University 
David Evans 

Education Policy Consultant 



Shaun R. Harper 
Assistant Professor, 

Higher Education Management, 
University of Pennsylvania 
Donald Heller 

Professor and Director, Center for 
the Study of Higher Education, 

School of Education, Pennsylvania 
State University 
Waldo Johnson 

Associate Professor, 

School of Social Service Administration, 
University of Chicago 
Scott Miller 

Director of Federal Relations, 
Pennsylvania Higher Education 
Assistance Agency 
Barmak Nassirian 
Associate Executive Director, 

American Association of Collegiate 
Registrars and Admissions Officers 
Raymund Paredes 

Commissioner, Texas Higher 
Education Coordinating Board 
Tom Wolanin 
Senior Associate, 

Institute for Higher Education Policy 




THE PELL INSTITUTE 

For the Study of Opportunity in Higher Education 

1025 Vermont Avenue, NW, Suite 1020 Washington, DC 20005 

P: (202) 638-2887 F: (202) 638-3808 I www.pellinstitute.org 



MAY 2009 



